Depth cameras (also known as “range cameras”) are devices which produce two-dimensional (2D) images—known as “depth maps”—showing the distance of points in a scene from a specific point of the depth camera. Several different technologies can be used to implement depth cameras; among the most widely used one might mention time-of-flight cameras, based on a principle similar to radar, and structured light cameras, which project an infrared pattern onto the scene, acquire an image thereof and deduce depth information using image processing. Depth cameras can be combined with conventional color (RGB—Red, Green and Blue) cameras to produce two-dimensional images containing both color and depth information, which can be referred to as “RGB-Depth”, or simply “RGBD” images. Combining a depth camera with a black-and-white camera is also possible, albeit less useful.
As illustrated on FIG. 1, a depth camera DC can be used to acquire multiple depth maps DM1, DM2, DM3 (“pure” depth maps or RGBD images) of a “physical” three-dimensional object PO from different positions. A reconstruction algorithm RA, executed by a computer, produces a three-dimensional reconstruction 3DR of the physical object taking the depth maps as inputs. Several suitable reconstruction algorithms are known in the art, see e.g. R. A. Newcombe et al. “KinectFusion: Real-Time Dense Surface Mapping and Tracking”, Symposium ISMAR 2011.
The reconstruction algorithm does not necessarily need to receive, at its inputs, the positions and orientations (“poses”) taken by the depth cameras to acquire the different depth maps, because this information—also called “extrinsic” calibration—can be deduced using the depth maps themselves. On the contrary, “intrinsic” calibration parameters of the depth camera—such as its focal length(s), principal point and possibly distortion parameters—do have to be provided to the reconstruction algorithm. The importance of the intrinsic calibration parameters is highlighted by FIG. 2, which shows different reconstructions of two three-dimensional objects—a figurine representing a rabbit on the top line, a cube on the bottom line—obtained by providing to the reconstruction algorithm different values of the focal length of the depth camera used to acquire the depth maps used as inputs. On both lines, the central image (in a frame) corresponds to the correct values of the focal length; images to the left and to the right correspond to increasingly incorrect values of this parameter. It can be seen that an incorrect calibration of the depth camera can lead to a completely meaningless reconstruction.
In some cases, calibration parameters provided by the manufacturer of a depth camera can be used for 3D reconstruction (this is the case of the above-referenced paper by R. A. Newcombe et al.), but if more accurate results are required, a dedicated calibration step has to be carried out. The problem of calibrating a depth (or a RGB-Depth) camera has been addressed e.g. in the following documents:    Herrera C., D., Kannala, J., Heikkilä, J., “Joint depth and color camera calibration with distortion correction”, TPAMI, 2012;    R. Y. Tsai, “An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision”, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, Fla., pp. 364-374, 1986;    Zhang, Zhengyou “Flexible Camera Calibration By Viewing a Plane From Unknown Orientations”, ICCV, 1999.
The first of these documents concerns specifically the case of a RGB-Depth camera, but can also be applied to other kinds of depth camera which also provides an intensity image of the scene; the other two concern the calibration of conventional cameras allowing no depth measurements, but their teaching also applies to RGB-Depth cameras, and to depth cameras acquiring a grayscale image of the scene over which an infrared pattern has been projected.
These approaches use a planar pattern of known 2D geometry, such as a chessboard with known distance between corners, several images of which are acquired using the RGB or RGB-Depth camera to be calibrated. These images allow estimating a projection matrix of the camera, from which calibration parameters are extracted.
These approaches are more robust than the use of constructor datasheets, but there is no guarantee that they provide calibration parameters sufficiently accurate for 3D reconstruction; for instance, an error on focal lengths can be compensated by a drift on extrinsic parameters, i.e. parameters expressing the position and orientation of the camera. Moreover, the obtained parameters are extremely dependent on the protocol used for image acquisition.
The paper by Popo Gui et al. “Accurately Calibrate Kinect Sensor Using Indoor Control Field” discloses a method of calibrating a depth camera by imaging a control field comprising a number of control points of known position. This method only applies to certain kinds of depth camera, and the control points have to be materialized by markers, which may be impractical.